Perceptions in a haptic system

ABSTRACT

A system providing various improved perceptions techniques for haptic feedback above interactive surfaces that require no contact with either tools, attachments or the surface itself is described. A range of receptors in a perceiving member which is part of the human body is identified to create substantially uniformly perceivable feedback. A vibration frequency that is in the range of the receptors in the perceiving member is chosen and dynamically altered to create substantially uniformly perceivable feedback throughout the receiving member.

RELATED APPLICATION

This application claims the benefit of the following four U.S.Provisional Patent Applications, all of which are incorporated byreference in their entirety:

1. Ser. No. 62/118,560, filed on Feb. 20, 2015.

2. Ser. No. 62/193,234, filed on Jul. 16, 2015.

3. Ser. No. 62/206,393, filed on Aug. 18, 2015.

4. Ser. No. 62/275,216, filed on Jan. 5, 2016.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to improved perceptiontechniques in haptic-based systems.

BACKGROUND

Multi-touch surfaces have become common in public settings, with largedisplays appearing in hotel lobbies, shopping malls and other high foottraffic areas. These systems are able to dynamically change theirinterface allowing multiple users to interact at the same time and withvery little instruction.

There are situations when receiving haptic feedback before touching thesurface would be beneficial. These include when vision of the display isrestricted, such as while driving, and when the user doesn't want totouch the device, such as when their hands are dirty. Providing feedbackabove the surface would also allow for an additional information channelalongside the visual.

A mid-air haptic feedback system creates tactile sensations in the air.One way to create mid-air haptic feedback is using ultrasound. A phasedarray of ultrasonic transducers is used to exert an acoustic radiationforce on a target. This continuous distribution of sound energy, whichwill be referred to herein as an “acoustic field”, is useful for a rangeof applications, including haptic feedback.

Accordingly, a system that provides various improved perceptionstechniques for haptic feedback above interactive surfaces and requiresno contact with either tools, attachments or the surface itself isdesirable.

BRIEF DESCRIPTION OF THE FIGURES

The accompanying figures, where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, together with the detailed description below, are incorporated inand form part of the specification, and serve to further illustrateembodiments of concepts that include the claimed invention, and explainvarious principles and advantages of those embodiments.

FIG. 1 is a representation of shapes made in a haptic system.

FIG. 2 is an illustration of an example series of five haptic controlpoints simultaneously produced in the plane of interaction.

FIG. 3 is a selection of acoustic field simulations where in the controlpoint is moved through the plane of interaction.

FIG. 4 is an illustrative view of unconstrained and constrainedtransducers.

Skilled artisans will appreciate that elements in the figures areillustrated for simplicity and clarity and have not necessarily beendrawn to scale. For example, the dimensions of some of the elements inthe figures may be exaggerated relative to other elements to help toimprove understanding of embodiments of the present invention.

The apparatus and method components have been represented whereappropriate by conventional symbols in the drawings, showing only thosespecific details that are pertinent to understanding the embodiments ofthe present invention so as not to obscure the disclosure with detailsthat will be readily apparent to those of ordinary skill in the arthaving the benefit of the description herein.

DETAILED DESCRIPTION I. Challenges of Creating Acoustic Fields in HapticSystems

It is known to control an acoustic field by defining one or more controlpoints in a space within which the acoustic field may exist. Eachcontrol point is assigned an amplitude value equating to a desiredamplitude of the acoustic field at the control point. Transducers arethen controlled to create an acoustic field exhibiting the desiredamplitude at each of the control points.

A. Human Hand Properties

Vibration is detected by mechanoreceptors within the skin. Themechanoreceptors within the skin are responsive to vibrations in therange 0.4 Hz to 500 Hz. The emitted ultrasound may be modulated in orderto create vibrations within the optimum frequency range detectable bythe human hand. By changing the modulation frequency, it may also changethe frequency of the vibration on the hand and this can be used tocreate different tactile properties. Modulating different focal pointsat different frequencies can give each point of feedback its ownindependent “feel”. In this way it is possible to correlate haptic andvisual feedback and also attach meaning to noticeably different texturesso that information can be transferred to the user via the hapticfeedback.

Specifically, when human skin interacts with the acoustic field,vibrations of the skin are interpreted by mechanoreceptors being excitedand sending signals to the brain via the nervous system. For example,the palmar surface of the hand has 4 different types ofmechanoreceptors, each of which responds to a different range offrequencies. The force required to trigger each of these receptorsvaries with the frequency of the vibration. For example, the Paciniancorpuscle has its lowest activation threshold at around 200 Hz, whilethe Meissner corpuscle is most sensitive between 10-50 Hz.

These receptors are distributed in differing densities throughout theskin. For example, a 200 Hz vibration will not feel as strong on thefinger tips as on the palm of the hand, due to there being a largerconcentration of the Pacinian corpuscle in the hand.

Ultrasound haptic feedback systems create a vibro-tactile sensation uponthe skin of a user of the system. The focused ultrasound creates enoughforce at the point of intersection to slightly displace the skin of auser. Typically, ultrasound haptic feedback systems use ultrasound witha frequency at or above 40 kHz, which is above the threshold forreceptors in the skin to feel. Therefore, a user can only detect theonset and cessation of such focused ultrasound. In order to provide asensation that is detectable by the receptors in skin, the focusedultrasound is modulated at a lower frequency, within the detectablerange of the receptors. This range is typically from 1 Hz to 500 Hz.

When creating a system with mid-air tactile feedback, it is important toselect the correct frequency of vibration for the part of skin that isbeing targeted. For example, if the hand is targeted, a vibrationfrequency of 110 Hz is a good choice as it can be felt across all partsof the hand, albeit to varying degrees of strength.

B. Transmitting Ultrasound Signals for Haptic Feedback

Tactile sensations on human skin can be created by using a phased arrayof ultrasound transducers to exert an acoustic radiation force on atarget in mid-air. Ultrasound waves are transmitted by the transducers,with the phase emitted by each transducer adjusted such that the wavesarrive concurrently at the target point in order to maximize theacoustic radiation force exerted.

By defining one or more control points in space, the acoustic field canbe controlled. Each point can be assigned a value equating to a desiredamplitude at the control point. A physical set of transducers can thenbe controlled to create an acoustic field exhibiting the desiredamplitude at the control points.

A side effect of this technique is that the ultrasound breaks down andcreates a sound at the modulation frequency. Therefore, when creatingtactile feedback with a 200 Hz modulation frequency, a 200 Hz sound isalso produced. This audible sound may be annoying to users and can be abarrier to ultrasound haptic technology being adopted.

The optimal conditions for producing an acoustic field of a singlefrequency can be realized by assigning activation coefficients torepresent the initial state of each transducer. However, in order tocreate haptic feedback, the field can be modulated with a signal of apotentially lower frequency. For example, an acoustic field of 40 kHzmay be modulated with a 200 Hz frequency in order to achieve a 200 Hzvibro-tactile effect. Methods to generate this vibro-tactile effect mayreduce audible content by smoothly interpolating the transduceractivation coefficients between discrete and disjoint sets of controlpoints, resulting in smooth sinusoidal amplitude variations at thecontrol point locations. These sinusoidal amplitude variations cause apure tone to be generated. A pure tone, although much lower inperceptual loudness than the disparate frequency content caused byabruptly changing the state of the transducers, remains audible.

It is known that the modulation waveform can be shaped to reduce thevolume of the audible noise that is created (as described for example inUK Patent Application No. 1415923.0). In general, reducing and avoidingsharp changes in pressure level at the focus will reduce the loudness ofthe audible sound. For example, modulation with a pure square wave willproduce a louder audible sound than modulation with a pure sine wave.

Further, in this acoustic field, one or more control points can bedefined. These control points can be amplitude modulated with a signaland as a result produce vibro-tactile feedback in mid-air. Analternative method to produce feedback is to create control points thatmay not be modulated in amplitude and instead move them around spatiallyto create spatio-temporal modulation that can be felt. Both methods maybe then used separately or together in order to produce sound anddifferent textures.

II. Creating Uniform Feeling in a Haptic System

A. Steps to Create Optimized Haptic Feedback

The steps to create optimized haptic feedback using multiplexedfrequencies include the following:

1. Knowing the distribution and frequency response range of receptors inthe skin, create uniformly perceivable feedback.

2. Selecting a frequency that is in the range of all receptors.

3. Optimizing the vibration frequencies to not just be perceivable, butthe best possible vibration frequency for a strong or high qualityfeeling.

4. Dynamically adjusting the vibration frequency to always be at theoptimal frequency.

5. Multiplexing multiple frequencies to create a vibration that providesan optimum level of strength and quality across the whole target area.

B. Optimization of Haptic Feedback

In order to optimize the quality and perceived strength of the feedback,it is possible to dynamically vary the vibration frequency as theinteraction is carried out. For example, in the situation of a tactilethreshold, where a hand passes through some fixed plane and has a lineof vibration created at the intersection of the hand and that plane, thefrequency of the vibration can be adjusted in real time to optimize forthe area of the hand currently being vibrated. One possibility is thatthe fingers receive a 100 Hz vibration while the center of the palmreceives a 200 Hz vibration.

This dynamic adjustment is also possible with multiple points offeedback. For example a mid-air array of buttons could be represented byone localized point of vibration in the air for each button. As a handmoves over the array of buttons to explore their position andorientation, the vibration frequency of each button could adjust in realtime to match the optimum frequency for the part of the hand it istargeted at.

There are also many situations where dynamic adjustment is not possibleor not desirable. For example, the tracking system may not besophisticated to determine discrete parts of the hand or the requiredprocessing power may be too high.

In these situations, it may be possible to multiplex frequencies toprovide some uniform coverage across the area of skin being targeted.For example, when targeting a hand, a point of feedback could bemultiplexed with both 100 Hz and 200 Hz. The palm would respond stronglyto the 200 Hz component while the finger tips would respond strongest tothe 100 Hz component. In so doing, a point of feedback that can beuniformly felt across the hand is established.

III. Creating Distinct Shapes and Corners

Using vibrations to generate mid-air haptic shapes leads to difficultieswith corners. Haptic edge detection has been shown to require highlylocalized skin displacement (stretching) and at the ultrasonicfrequencies currently used this is not possible. Edges can be reliablydetected, but corners are not large enough features to be easilyrecognizable.

The use of points of high pressure in the ultrasonic field conveyvibrations and are called “control points.” They provide local feedbackin an area around a wavelength in a small diameter (such as 8.6 mm @ 40kHz). The control points may be arranged in 3D space programmatically tocreate the feeling of a shape in space.

There is a de facto maximum density for control points of about twowavelengths (˜2 cm @ 40 kHz) apart, as diminishing returns in fidelityare exchanged for increased noise. This means that when using thecurvature-dependent control point density, edge fidelity must besacrificed in order for corner points to be noticeable. In manysituations, even this is not enough to render corners in haptic shapesdistinguishable.

To enhance corners, it is possible to warps the edges of the shapeinwards to emphasize corners in space, creating a haptic representationaccentuated in a fashion that enables the perception of corners fromspatial cues.

Turning to FIG. 1 , shown in the left box 10 is a shape withoutcorner-enhancing warping. Shown in the center box 20 of FIG. 1 is acorner-enhancing warp function applied to highlight the cornershaptically. Shown in the right box 30 of FIG. 1 is a function wherefurther warping has been applied, showing that the effect is tunabledepending on the circumstances and the effect desired.

Once the shape warping has been achieved, other techniques for enhancingcorners can also be used. Specifically, the curvature dependent controlpoint density and the rotation of the points in time may be altered toproduce a desired effect. This shape warping may be applied to sectionsof 3D geometry to create haptic 3D geometry with distinct corners toincrease haptic fidelity. This may also be applied as a process tohighlight salient features in shapes for attracting attention.

IV. Creating Pulsing Points

Due to awareness of the de facto limit of control point definition anddensity, it is possible to create pulsing points that are hapticallypleasing. By forcing control points close together, they merge andbecome a smaller, weaker vibration. By rotating them and bringing themcloser and further apart, a localized pulsing sensation can be generatedthat can haptically give a standby or ready prompt.

Turning to FIG. 2 , shown is an illustration of an example series offive control points simultaneously produced in the plane of interaction.The points in the figure have a diameter of a wavelength. The fivepoints spin quickly so they are not distinguishable. In the left panel40 of FIG. 2 , the five points spin and are far enough apart for them tobe perceived as a single large haptic point. The central panel 50 showsthat as the points orbit closer together the haptic point shrinks andbecomes weaker. In the right panel 60, the five control points havemerged to become a single control point. The process is then reversed toincrease the size and strength of the haptic point and this system isthen cycled to generate a pulsing sensation. This results in a pointwhich feels larger and smaller with time to produce a hapticallypleasing pulsing effect. As such, FIG. 2 shows a set of points, spinningin a circle with the diameter of the circle growing smaller/bigger overtime. (An alternative method to the spinning circle is to move the focusbetween two locations.)

Turning to FIG. 3 , shown is a selection of acoustic field simulationswhere in the control point is moved through the plane of interaction(shown approximately by the inset black bordered box). The small filledblack circles along the bottom edge of each figure represent transducerelements that have been configured to reproduce a control point. In theleft panel 70 the control point is created below the plane ofinteraction, resulting in no focusing in the interaction space. As thefocus moves up, the interaction space contains the control point asshown in the central panel 80. Finally, in the right panel 90 the focushas moved up through and out of the interaction space. This process isthen reversed and cycled to produce the pulsing sensation. Thus, bymoving the control point backwards and forwards through the inset box, apulsing sensation can be produced throughout the central region. Userdetection in this scenario can be much more crude, e.g. a light sensorin the central area.

The objective as shown in these figures is to create a “pulsating”sensation. “Pulsating” is defined to be a sensation that grows bothstronger/weaker and bigger/smaller over time. (By comparison, a simplemodulation only grows stronger/weaker over time.)

Further, FIG. 3 shows one possible example, where the focus is linearlyinterpolated between two positions, one vertically below the interactionzone and the other vertically above. As the focus moves up and down, thesensation experienced in the interaction zone grows smaller/larger asthe ultrasound focus is cone shaped. (in the figure, larger on the leftimage 70 and right image 90, smaller in the middle image 80). This alsohas the effect of making the sensation stronger/weaker as the strengthdrops off moving away from the optimal focus (shown in the middle image80).

Moving the focus up and down through the interaction zone also has abenefit with the tracking system. Less accuracy is required in thevertical direction. As shown in FIG. 3 , if interaction zone is moved upor down, it still experiences a sensation that grew bigger/smaller andstronger/weaker over time. This means that less accuracy in the verticalaxis from the tracking system is needed, allowing for the use of acheaper tracking system.

Alternatively, varying the focusing location can create a pulsingsensation. This has the effect of alternately focusing and defocusingthe control point, which generates a lower fidelity pulsing. Althoughthis is less effective, it may be potentially useful for situations inwhich a pre-baked, offline response that does not need active sensing isrequired.

V. Combining and Designing Audible and Haptic Feedback

A. Designing the Audible Feedback

As the audible sound is created by the modulation waveform, it ispossible to design the sound that is produced. For example, rather thanmodulating the focused ultrasound with a pure waveform, modulating itwith the waveform of a “click” sound will result in an audible “click”sound being produced. Thus, the modulation waveform can be designed anddynamically changed to produce any audible sound.

When using focused ultrasonic carrier waves, the audible sound isproduced most intensely at the focus and is directional. To the user,this means the sound appears to originate from the focus. This may be ofgreat use in a haptic system. For example, a mid-air button may becreated by focusing the ultrasound onto the user's fingertip. Theultrasound can then be modulated with the waveform of a “click” sound.The user would perceive both a haptic click sensation and an audible“click” sound originating from their finger tip. Thus, both the hapticand audio feedback are created at the same location in mid-air.

B. Separating Audio and Haptic Feedback

The modulation waveform that provides the optimum haptic feedback willoften differ from that which provides the optimum audible sound. Forexample, a modulation waveform that creates a pleasing “click” sound mayprovide a very weak tactile sensation or a modulation waveform thatprovides a strong tactile sensation may provide an annoying sound. Whendesigning for a combination of haptic and audio feedback, it istherefore necessary to make trade-offs between the two.

A solution to this is to create multiple points of focus within theacoustic field. Each of the points can be purposed with either creatinga haptic effect or creating audible feedback. In the simple button clickexample, one point can be positioned on the fingertip to create thehaptic effect, while another can be positioned elsewhere in the acousticfield to create the audible sound. In this scenario, the audible pointwould be positioned to avoid contact with the user, and thus would notbe able to be felt.

C. Auditory Masking of the Haptic Effect by the Auditory Sound

Auditory masking of a sound occurs when the perception of one sound isaffected or covered up by another sound. As the sound from the focusedultrasound is directional, the audible point can be positioned anywherealong the path between the transducers and the user's head or ear whereit would then maximize the perceived volume of that sound. By contrast,the audible sound created by the haptic point would be reflected off thefinger and away from the user. It would therefore be perceived asquieter. Thus, the audible sound from the haptic point will be masked bythe audible sound from the audible point and the user would only be ableto hear the audible point.

Haptic and audible points can each have their own separate modulationwaveforms. This allows each type of point to be modulated with theoptimum waveform for their individually desired effect. Indeed, systemsare not limited to having only two simultaneous points. It is possibleto have many independent and simultaneous points of both haptic feedbackand audible feedback.

VI. Spatio-Temporal Modulation in Haptic Systems

A. Absolute Phase Offset

In order to create smooth transitions between any two complex spaces oftransducer activation coefficients they should differ as little aspossible. All sets of transducer activation coefficients have one sparedegree of freedom: their absolute phase relative to some otheractivation coefficient pattern. By tying both to some arbitrary gaugepoint—such as making both as zero phase offset as possible—this reducesto a minimum the frequency shifts in the transducers required to movebetween the two patterns.

This is illustrated in FIG. 4 , which shows an illustrative array of twotransducers. The focusing of this array is due to the relative phaseoffsets in the wave between both transducers, which is described by theangle between them. In the unconstrained example, on the left, the phaseat time t=0 100 has to change considerably in order to reach the complexactivation coefficients defined for time t=1 110. This induces transientbehavior, frequency shifting and power inefficiency when scaled acrossmany transducers. In the constrained example on the right however, thesum of the transducer coefficients has been constrained to the real lineat t=0 120 and t=1 130, facilitating a small change in angle to obtainthe appropriate relative phase.

Summing the complex-valued transducer activation coefficients of anyparticular pattern can be used to produce an average. Once computed, thecomplex conjugate of this average can be taken and made into a unitamplitude complex value. Then by multiplying each transducer activationcoefficient with this value, the average phase offset becomes zero. Thisuses the spare degree of freedom to linearly minimize the changesexperienced by each transducer, pushing the differences between thecomplex phase spaces of the transducer activation coefficients forinfinitesimally different patterns towards zero, a key necessity forreducing abrupt changes in the acoustic field and transducer powerconsumption generally.

B. Spatio-Temporal Modulation of Points, Lines and Shapes

Creating and destroying control points (increasing amplitude from ordecreasing amplitude to zero) is associated with noise as the amplitudevaries. To maximally reduce noise, non-modulated control points may becreated and move around on sets of curve pieces defined parametrically.While non-modulated control points will maximally reduce noise, “lessmodulated” control points may also reduce noise to a lesser extent (thatis modulating the amplitude between 0.5-1 rather than 0-1).

The defined curves will either be closed, where the points willcirculate continuously on the curve, or open, where the points willreverse direction. Alternatively, the points may “disappear” uponreaching the end of the curve. When multiple points highlight a singleopen curve it may be useful for them to swap positions when they becomeclose enough to be perceived as a single point to prevent any perceptualor physical drop in output.

This open or closed path curve, which may be implemented as athree-dimensional spline curve, is the fundamental building block of asystem for creating spatio-temporally modulated feedback. Many of thesecurves running through three-dimensional space may function as contoursand be used to create the impression of a shape or surface. A circularpath with very small radius may be used to create the impression of apoint.

Since this technique requires less focusing time to produce the sameresponse, larger areas may be haptically actuated. For instance,‘painting’ an area using a control point may be used to create a widerregion in space that can be felt to stimulate more receptors in theskin.

Due to the wide area that can be actuated, feedback may be removed fromplaces to create an impression of the negative space. For instance, acircle with a piece missing may be created to represent a hapticfiducial or generate an area of feeling with a conspicuously missingregion. By emitting haptics around a space, an area may be highlightedwithout having to require it to be felt directly, for instance when thearea in question must be kept clear of hands or limbs for visibilityreasons.

C. Parametric Sound from a Control Point

The haptic sensation from the control point is generated through theaction of the spatio-temporal strobe effect. Since the source of thehaptic effect is distinct and different from amplitude modulation,amplitude modulation may be used with non-haptic audible content whilealso creating high-quality haptic sensation at the same time in the samepoint, line or shape.

The ultimate goal of the technology is to produce a silent operation.Audible output is correlated with temporal changes in the acousticfield, which must therefore be as smooth and reduced as possible. Tocreate this effect, it is possible to move the control points of theultrasonic focused patterns around smoothly using high speed updates(preferably greater than 2 kHz) to produce sensations at target points,lines and shapes rather than creating them unchanging in time with muchhigher intensity ultrasonic devices or manipulating their amplitude intime.

Modulating in this manner may use much lower strobing frequencies thanwould be expected by simply considering mechanoreceptors or the densityof focused power in time. Therefore, combining both spatio-temporal andamplitude modulation of control points at once can also be used producestronger haptic feedback. Different rhythms of control point motion mayalso be used to provide different textures, and thus the amplitudemodulation can provide texture to the spatio-temporally modulatedcontrol points or vice versa.

VII. Frequency Control Using Self-Intersecting Curves

Creating a point and moving it without modulating in amplitude mayhaptically actuate a path in the air. By making the point follow a pathrepeatedly at a given speed and a constant frequency, a haptic effectmay be generated on the path in the air. Making the path longerincreases the path distance and thus the speed needed for the point toachieve a given frequency. Consequently, this reduces the power densityavailable for creating feedback. Multiple points may haptically actuatea path to create a more even distribution around the path at a givenfrequency. But this reduces the power that can be brought to bear atthese points. A further significant limitation is that the path must beclosed or audible sound will result due to the discontinuities involvedin open paths, such as for instance, a distinct point or line segment.

One way to overcome these issues is to slow or accelerate the points asthey move along a curve. However, when used with spatio-temporalmodulation this has limitations both in that the frequency cannot bedifferent for different points along the path, and that if the pointslows too greatly it becomes less perceptible as it moves out of therange of frequencies perceptible haptically. Conversely, if the pointmoves too quickly along its path, it can create further air disturbancesand thus audible noise.

Instead of simply changing the speed of the points, an increase in theamount of power at given points on the path can be realized through theconstruction of a self-intersecting curve that crosses over itself oneor more times in space. In the local neighborhood of the intersectionpoints, harmonics of the base path frequency can be expressed. Thischange in the frequency behavior also increases the power at thecrossing point, enabling a rich palette of behaviors to be realized inthe crossing region that are in contrast to other locations along thepath. This intersection may also be engineered to make only theintersection within the detectable frequency range of human touch, oronly the intersection outside the frequency range and thus hapticallyundetectable. It also enables both the specific and broad targeting ofdifferent mechanoreceptors in the skin. In many scenarios, the absenceof haptic feedback in an area surrounded by haptic feedback may be feltmore strongly than the feedback itself.

Multiple paths that intersect one or more times may also be created;this will produce a pattern of haptic cadence that embodies a particularrhythm or texture. This does not necessarily mean that the path or pathsshould be repeatable or the crossing point of the curve at the sameposition each time. In some cases, due to the movement of theindividually-considered points being too fast or weak to feel, it can beengineered such that the crossing point or points are hapticallyhighlighted due to the self-intersection. In each case, these multiplecrossing or self-intersecting curves may contain either or both pointregions and path segments wherein the curves occupy the same space.

The points moving along the curves defined at a precise location have areal physical size. The abstract notion of “curves” does not reflect thesize of the points, so in many cases the abstract curves do not have tointersect, but instead the area of effect of the point merely has to beoverlapping to realize an increase in average power and frequency. Dueto this, a set of paths or self-intersecting curves can potentially leadto unanticipated haptic results. Therefore, the produced haptic patternsmay be best visualized using a frequency/occupancy graph and such afrequency/occupancy graph can be transformed to and from arepresentative haptic curve.

VIII. Conclusion

The various features of the foregoing embodiments may be selected andcombined to produce numerous variations of improved haptic-basedsystems.

In the foregoing specification, specific embodiments have beendescribed. However, one of ordinary skill in the art appreciates thatvarious modifications and changes can be made without departing from thescope of the invention as set forth in the claims below. Accordingly,the specification and figures are to be regarded in an illustrativerather than a restrictive sense, and all such modifications are intendedto be included within the scope of present teachings.

The benefits, advantages, solutions to problems, and any element(s) thatmay cause any benefit, advantage, or solution to occur or become morepronounced are not to be construed as a critical, required, or essentialfeatures or elements of any or all the claims. The invention is definedsolely by the appended claims including any amendments made during thependency of this application and all equivalents of those claims asissued.

Moreover in this document, relational terms such as first and second,top and bottom, and the like may be used solely to distinguish oneentity or action from another entity or action without necessarilyrequiring or implying any actual such relationship or order between suchentities or actions. The terms “comprises,” “comprising,” “has”,“having,” “includes”, “including,” “contains”, “containing” or any othervariation thereof, are intended to cover a non-exclusive inclusion, suchthat a process, method, article, or apparatus that comprises, has,includes, contains a list of elements does not include only thoseelements but may include other elements not expressly listed or inherentto such process, method, article, or apparatus. An element proceeded by“comprises . . . a”. “has . . . a”, “includes . . . a”, “contains . . .a” does not, without more constraints, preclude the existence ofadditional identical elements in the process, method, article, orapparatus that comprises, has, includes, contains the element. The terms“a” and “an” are defined as one or more unless explicitly statedotherwise herein. The terms “substantially”, “essentially”,“approximately”, “about” or any other version thereof, are defined asbeing close to as understood by one of ordinary skill in the art. Theterm “coupled” as used herein is defined as connected, although notnecessarily directly and not necessarily mechanically. A device orstructure that is “configured” in a certain way is configured in atleast that way, but may also be configured in ways that are not listed.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims. In addition, in theforegoing Detailed Description, it can be seen that various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed embodiment. Thus the following claims arehereby incorporated into the Detailed Description, with each claimstanding on its own as a separately claimed subject matter.

We claim:
 1. A method comprising: i) producing an acoustic field from atransducer array having known relative positions and orientations; ii)defining a plurality of control points wherein each of the plurality ofcontrol points has a known spatial relationship relative to thetransducer array; iii) engaging in rhythms of motion of at least one ofthe control points that correlated with temporal changes in the acousticfield to produce a desired audible sound.
 2. The method as in claim 1,wherein the acoustic field is produced by a mid-air haptic feedbacksystem.
 3. The method as in claim 1, wherein the desired audible soundis perceived to be originating from the at least one of the plurality ofcontrol points.
 4. The method as in claim 1, further comprisingmodulating the acoustic field associated with at least one of theplurality of control points to produce localized haptic feedback in thesame general location as the desired audible sound.
 5. The method as inclaim 4, wherein the localized haptic feedback is separated from thedesired audible sound wave so that the desired audible sound producesperceivable haptic feedback below a preset threshold.
 6. The method asin claim 4, wherein the desired audible sound is perceived by a user tobe louder than the audible sound produced by the localized hapticfeedback.
 7. A method comprising: i) producing an acoustic field from atransducer array having known relative positions and orientations; ii)defining a plurality of control points wherein each of the plurality ofcontrol points has a known spatial relationship relative to thetransducer array; iii) modulating a position of at least one of theplurality of control points to produce a desired audible sound; and v)creating a plurality of points of focus within the acoustic field,wherein a haptic effect is at a first point of focus and a desiredaudible sound is at the second point of focus; vi) creating a pulsingsensation by manipulating location of the plurality of control points sothat the pulsation sensation grows both stronger/weaker andbigger/smaller over time.
 8. The method as in claim 7, wherein theacoustic field is produced by a mid-air haptic feedback system.
 9. Themethod as in claim 7, wherein the first point of focus is in a differentlocation than the second point of focus.
 10. The method as in claim 7,wherein the desired audible sound is perceived to be originating fromthe at least one of the plurality of control points.
 11. The method asin claim 7, further comprising modulating the acoustic field associatedwith at least one of the plurality of control points to producelocalized haptic feedback in the same general location as the desiredaudible sound.
 12. The method as in claim 11, wherein the localizedhaptic feedback is separated from the desired audible sound wave so thatthe desired audible sound produces perceivable haptic feedback below apreset threshold.
 13. The method as in claim 11, wherein the desiredaudible sound is perceived by a user to be louder than the audible soundproduced by the localized haptic feedback.
 14. The method as in claim11, wherein the desired audible sound is perceived by a user primarilyfrom the first point of focus.
 15. The method as in claim 11, whereinthe desired audible sound is perceived by a user primarily from thesecond point of focus.